INSTRUCTIONS FOR NCDC_CNV.EXE SOFTWARE, RELEASE DATE 9/17/08

Developed and distributed without charge by Russell F. Lee of RF Lee Consulting, Charlotte, NC.

PURPOSE:  This software converts selected parameters from the NCDC comma-separated values (CSV)
"abbreviated" hourly surface data files, the abbreviated DATSAV3 files, and the abbreviated ISHWO 
(Integrated Surface Hourly Weather Observations CDROM) into the SAMSON format.  It will concatenate 
multiple files, so that twelve files (or more) of monthly data can be input to obtain a year of 
data in SAMSON format.  

In addition, a second SAMSON format file is produced with missing data filled as described in 
"Procedures for Substituting Values for Missing NWS Meteorological Data for Use in Regulatory Air 
Quality Models" by Dennis Atkinson and Russell F. Lee (1992) which is available on 
http://www.RFLee.com and on the U.S. EPA SCRAM website at http://www.epa.gov/scram001.  The 
"Objective" procedure described in this document is used with one exception.  Up to three missing 
hours of temperatures are interpolated instead of only one hour.  

****************************************************************************************

				********************************
				*** DOCUMENTATION OF UPDATES ***
				********************************

9/17/2008 Bug fix:  The previous version would unexpectedly stop with an error message that the met
	data was out of sequence in some ISH data sets.  This has been fixed

12/18/2007 Allows the use of post-2005 comma-delimited ISH data from NCDC Online Store.  Pressures are 
	set to missing until I verify the units for some of the data sources, and correct the conversions
      if necessary.

10/3/2007 Change to allow NCDC_CNV to select hourly data up to 30 minutes before the hour instead of 20.

6/14/2007 Fixed a problem where an occasional non-missing data would appear in the SAMSON file
	as missing and/or the hours would get shifted by one hour in certain situations.

9/19/2006 IMPORTANT CORRECTION:  Integrated Surface Hourly Weather Observations CD-ROM outputs data
	with the time in UTC (GMT) instead of local standard time.  The NCDC_CNV software has been 
	corrected to make the time conversion to local standard time in the SAMSON format output.

One modification from previous version.  NCDC_CNV is now able to select hourly data
	up to 20 minutes before the hour instead of 10 minutes.

4/15/2006 Update:  Improvements over previous versions
	>  Primary improvement is that the columns of the comma-delimited (CSV) NCDC data are now 
	   identified by strings in the header instead of column number.  This should make the code
	   capable of accomodating all previous versions and future changes to that format, as well.

	>  Changed missing data code for wind direction from 990 to 999.  This has no practical impact,
	   but was done for technical consistency.

	>  Code to identify unusual times for the official hourly observation (ABBRDS only--see below), 
	   except for the 0 to 10 minutes before the hour, has been removed because problems in the code 
	   and the difficulties in getting the problems resolved.  The option to select hourly data
	   that are identified as between 1 and 10 minutes before the hour is still available, as it 
	   has been in earlier versions.  See instructions below for details.  (THIS FEATURE WAS 
         REINSTATED LATER--SEE INSTRUCTIONS BELOW FOR LINE 1, ITEM 14.)

********************************************************************

				*******************
				*** DESCRIPTION ***
				*******************


The following parameters are converted:
	year
	month
	day
	hour
	minute
	sky conditions (alpha-numeric codes are converted to sky cover and ceiling height)
	wind direction
	wind speed
	temperature in Fahrenheit
	temperature in Celsius (Temperature in degrees Celsius is used. If it is missing, temperature
					in degrees Fahrenheit is converted and substituted)
	atmospheric pressure  (conversion temporarily disabled)
	report type (CSV data only)	(Only the hourly observations are used: report type='AA';
					hourly observations are inferred by the time in the other
					formats--see instruction 14 below)
	hourly precipitation
	relative humidity

Note that precipitation and relative humidity ARE NOW INCLUDED in this version of the program.  

Other parameters that are not used in models such as AERMOD and ISCST3 are ignored and coded as 
"missing" in the SAMSON format output.

*********************************************************************

				********************
				*** INSTRUCTIONS ***
				********************

To use this program, prepare a control file called FILELIST.INP.  ***NOTE THAT THE FORMAT HAS CHANGED FROM
SOME PREVIOUS VERSIONS OF THIS PROGRAM.***

!The control file must be named 'FILELIST.INP'.
The first line must contain the following items, separated by one or more spaces:

LINE 1:	1)	Filename of the output file converted to SAMSON format.  
			Missing hours are not filled in this file.
		2)	Name of output file with missing data filled per 
			Atkinson & Lee.
		3)	Station ID (5-digit number). This number will appear in the output SAMSON file.
		4)	Time zone (EST = -5, CST = -6, MST = -7, PST = -8, etc.
		5)	'N' or 'S' to indicate North or South latitude.
		6)	Latitude--whole degrees portion.
		7) 	Latitude--minutes portion.
		8)	'E' or 'W' to indicate East or West longitude.
		9)	Longitude--whole degrees portion.
		10)	Longitude--minutes portion.
		11)	Elevation of station above mean sea level, in meters.
		12)	Type of input data:
				ISH, ish: NEW!  Use for ISH CDROM ABBREVIATED DATA (I.E., CDROM DATA EXTRACTED USING
					THE "SPACE-DELIMITED" OPTION.  The decoding is similar to ABBRDS, except
					that times are assumed to be in GMT and are converted to local time for the 
					SAMSON format.  Also, the observation time may be specified up to 20 minutes 
					before the hour in this version for both ISH and ABBRDS, instead of the limit 
					of 10 minutes in earlier versions.  
				CSV, csv: Comma-separated values from NCDC online store.  This accommodates 
					all known variants of the format, and will likely accommodate future 
					ones as well.
				ABBRDS, abbrds: Abbreviated DATSAV3 file.  Do not use the ABBRDS specification
					for the optional abbreviated output from ISH data set, because times 
					will not be converted from GMT.  NOTE:  IF YOU HAVE ABBREVIATED DATSAV3 DATA
                              THAT IS IN "GMT" INSTEAD OF LOCAL TIME, USE "ISH" FOR THE INPUT DATA TYPE.

				REMINDER:  ISH is for data with times in GMT; ABBRDS is for data with times in Local Standard Time

            13)	NORMAL or SUBSLP.  NORMAL gives the SAMSON format the station pressure.  
				SUBSLP substitutes sea level pressure for station pressure.  CAUTION: Use
				SUBSLP only for stations fairly near sea level, when the station pressure is missing.
            14)	Code to identify minutes value(s) used for regular hourly (not special) data.  
				This is only used for space-delimited data (ABBRDS), but must always be present.
				For most data sets, the value is 0 (zero).  However, some data sets use the actual
				observation time, which may be up to 20 minutes before the hour.  If this is the
				case, a value of 40 is suggested.  Be aware, however, that this may cause an 
				occasional special observation just before the hourly observation may be used 
				instead of the hourly observation.  If you know that observations in your data set
				are taken at or after, for example, 55 minutes after the hour, then it would be
				wise to set the value to 55 to help avoid reading a special observation as though
				it were an hourly observation.  Values MUST be between 40 and 59, or 0.  Values
				less than 40 but greater than 0 will be reset to 50.  

LINE 2:	City or location name, up to 22 characters.

LINE 3:	State or province abbreviation, 2 characters.  2-letter country abbreviation, may be used, 
		if desired.

LINES 4 through N+3	List of filenames of N input files to be converted.  These will be concatenated,
		in the order listed, into the output file.


Example 1 of the first line (for a CSV abbreviated format from NCDC):
EWB9801.SAM  EWB9801-F.SAM 14734 -8 N 41 41 W 70 58 24 CSV NORMAL 00

Example 2 (for an Abbreviated DATSAV3 file):
EWB9801.SAM  EWB9801-F.SAM 14734 -8 N 41 41 W 70 58 24 ABBRDS NORMAL 50

The second line contains the city (or place) name.  Up to 22 characters are read.

The third line contains the two character state or province abbreviation.  For locations outside the 
U.S. and Canada, you may use a two-letter country abbreviation.  The information in the second
and third lines are merely output in the headers of the SAMSON file to help the user identify the file.

The fourth line contains the filename of the first data file to be read.

If a second file is to be read and concatenated to the first in the output, that filename goes on the
fifth line.

The program will continue converting and concatenating files until it gets to the end of the list.  A
common use will be to convert a year of data, twelve monthly files, into a single output file. To do
this, the fourth through the fifteenth lines must contain the twelve monthly files, in the correct 
sequence. 

When FILELIST.INP has been prepared, put it, NCDC_CNV.EXE and the files to be converted in the same folder,
and double click on NCDC_CNV.EXE.

This program is offered without charge on an as-is basis.  If you find any errors, or have suggestions for 
improvements, please feel free to email me at Russell.Lee@RFLee.com.  I would also like to know if you find it 
useful.

All parameters are included for the header of the SAMSON output.  It is no longer necessary to add any of 
these manually as with earlier versions.  Note that even though the station ID appears in an Abbreviated
DATSAV3 file, the ID you enter in line one of the control file will be the one that is put in the header
of the SAMSON-format output file.


I do update this file when required, so please check my website to be sure you have the most up-to-date version.

Russ Lee
9/17/08

	